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PRACTICAL  CONSIDERATIONS  FOR  FIXED  WING  HELMET-MOUNTED  DISPLAY 

SYMBOLOGY  DESIGN 

Eric  E.  Geiselman 
Air  Force  Research  Laboratory 
WPAFB,  Ohio 

Helmet-mounted  displays  (HMDs)  may  become  a  primary  source  of  head-up  information  in  future  tactical 
aircraft.  In  evaluations,  HMDs  have  produced  improved  situation  awareness  and  mission  performance.  To 
realize  this  improved  performance,  HMD  equipped  pilots  spend  significantly  more  time  looking  off-axis, 
away  from  traditional  displays.  Accordingly,  it  is  important  that  HMD  information  be  designed  so  targeting 
and  ownship  status  symbology  is  effective  and  safe.  The  U.  S.  Air  Force  Research  Laboratory  is  working  to 
develop  an  optimized  HMD  symbology  set.  Toward  this  end,  symbol  designs  must  accommodate  the 
conditions  of  intended  use,  technology  limitations,  and  integration  with  other  displays.  This  paper  discusses 
efforts  to  design  target  tracking/location  and  ownship  status  symbologies  for  the  day/night  all  weather 
fixed-wing  tactical  aircraft  application.  Design  principles  derived  from  both  laboratory  and  flight  test  are 
presented.  These  principles  relate  to  symbology  frame  of  reference,  orientation,  compression,  and  line  of 
sight  mechanization.  Evaluation  methodologies  are  also  discussed. 


INTRODUCTION 

Helmet-mounted  display  (HMD)  technology  will  be  fully 
integrated  into  next-generation  tactical  aircraft.  Although 
laboratory  research  and  flight  evaluations  have  consistently 
demonstrated  the  potential  for  HMDs  to  enhance  tactical 
situation  awareness  (SA)  and  improve  mission  performance, 
the  exact  utility  of  HMD  technology  can  not  yet  be  stated 
confidently.  On  one  end  of  the  spectrum,  the  HMD  may 
simply  provide  an  aimsight  reticle  and  symbology  for 
purposes  of  target  cueing  and  sensor  guidance.  The  other  end 
of  the  spectrum  sees  the  HMD  as  a  complete  replacement  for 
the  head-up  display  (HUD)  and  the  primary  source  of  all  head- 
up  information  (as  in  an  encapsulated  cockpit).  Because  each 
end  of  the  spectrum  has  its  respective  advantages  and 
disadvantages,  the  evolution  of  the  HMD  will  probably  lie 
somewhere  between  these  extremes.  The  challenge  to  the 
designer  tasked  with  determining  the  information  content  and 
functionality  of  the  HMD  is  to  ensure  that  the  technology 
provides  the  correct  information  for  the  pilot  at  the  correct 
time.  This  is  achievable  through  a  systematic  design  and 
validated  evaluation  approach. 

This  paper  describes  ongoing  efforts  of  the  Air  Force 
Research  Laboratory  (AFRL)  Visual  Display  Systems  Branch 
toward  optimizing  the  HMD  presented  information  in  order  to 
enhance  mission  performance  and  survivability  for  near-term 
HMD  technology  applications.  The  rationale  and  associated 
design  decisions  presented  here  are  based  on  previous 
research,  flight  test  feedback,  and  the  hardware  configuration 
of  the  first  HMDs  likely  to  be  operationally  fielded.  It  is 
intended  also  that  the  design  principles  presented  here 


generate  research  interest  and,  maybe  most  importantly, 
initiate  research  with  opposing  hypotheses.  The  following 
paragraphs  first  discuss  several  generalizations  we  use  as  the 
basis  for  HMD  symbology  design.  Next,  sample  symbology 
candidates  and  the  principles  under  which  they  were  designed 
are  presented.  Included  is  HMD  resident  target  location  and 
ownship  status  symbology.  Finally,  several  recommendations 
related  to  the  empirical  evaluation  of  HMD  symbology  are 
offered. 

HMD  Functionality 

The  primary  purpose  of  the  HMD  is  to  provide  target 
acquisition  information  to  the  pilot  and  for  the  pilot  to  provide 
target  cueing  information  to  the  aircraft.  HMD  resident 
information  should  first  be  designed  to  get  the  pilot’s  eyes  on 
a  target  and  lead  a  sensor  to  a  point  of  interest  (POI).  In 
parallel  arise  both  the  capability  and  perhaps  the  need  to 
provide  other  types  of  information  via  the  HMD.  For  instance, 
the  HMD  affords  the  unique  capability  to  present  ownship 
status  information  (including  airspeed,  altitude,  heading,  and 
attitude)  to  the  pilot  regardless  of  head  location  or  movement 
relative  to  the  aircraft  axes.  Ownship  status  presented  to  the 
pilot  off-axis  (other  than  along  the  aircraft  flightpath)  may  be 
very  useful  in  degraded  visual  conditions  and  at  night. 

A  primary  HMD  symbology  development  objective  is  the 
determination  of  how  different  information  categories  will 
work  together,  as  well  as  with  other  cockpit  displays.  The 
HMD  offers  a  new  coordinate  reference  frame  for  which 
information  needs  is  related.  This  is  both  a  challenge  and  an 
opportunity  for  the  designer  to  exploit  the  unique  symbology 
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functionality  afforded  by  a  head-coupled  coordinate  reference 
frame. 

Previous  research  and  flight  test  feedback 

From  a  review  of  previous  research  and  flight  test 
feedback,  the  following  conclusions  are  reasonable: 

1)  HMD  presented  information,  compared  to  HUD-only 
information,  enables  (or  compels)  pilots  to  look  farther 
off-axis  (off-boresight)  for  longer  periods  of  time  during 
air-to-ground  and  air-to-air  tasks.  (Osgood,  Geiselman  and 
Calhoun,  1991;  Geiselman  and  Osgood  1994). 

2)  Effects  of  duration  and  angle  off-axis  are  seen  independent 
of  information  category  (target  information  vs.  ownship 
status)  (Geiselman  and  Osgood,  1994;  Geiselman  and 
Osgood,  1995). 

3)  Pilots  prefer  off-axis  ownship  status  information  be 
included  within  the  HMD  symbology  set.  (Osgood  et  al, 
1991;  General  Dynamics,  1992;  Boehmer,  1994; 
Geiselman  and  Osgood,  1994;  Osgood  and  Chapman, 
1997;  Fechtig,  Boucek,  and  Geiselman,  1998). 

4)  Specific  symbology  formats  intended  for  use  in  the 
air-to-air  arena  should  be  designed  to  minimize  the  visual 
area  they  occupy  (Fechtig,  et  al,  1998). 

It  appears  that  any  information  presented  on  the  HMD 
may  compel  the  pilot  to  look  farther  off-axis  for  a  longer 
period  compared  to  the  same  task  performed  without  HMD 
presented  information.  This  may  result  in  increased  targeting 
SA,  but  the  effect  on  the  spatial  orientation  component  of 
overall  SA  is  not  yet  known.  It  is  important  that  we  avoid 
increasing  targeting  performance  only  to  accrue  a  cost 
elsewhere.  Based  on  these  findings,  the  symbology  design 
principles  presented  in  the  following  paragraphs  have  been 
derived.  Once  these  principles  have  been  validated,  along  with 
others,  it  is  intended  that  they  be  included  in  design  guidance 
documentation  such  as  Mil-Std-1787C. 

TARGET  LOCATOR  LINE  DESIGN 

The  purpose  of  the  target  locator  line  (TLL)  is  to  indicate 
the  relative  azimuth  and  elevation  vector  between  a  fixed  point 
of  reference  on  a  display  field  of  view  (FOV)  and  some  POI 
outside  the  display  FOV.  Locator  lines  have  been  used  with 
great  success  as  a  HUD  convention  as  well  as  during  HMD 
demonstrations  and  operational  flight  test. 

A  properly  designed  and  implemented  TLL  symbology 
can  help  the  pilot  to  quickly  and  intuitively  perform  a  visual 
location  task.  Design  features  can  be  added  to  the  locator  line 
to  provide  additional  information  such  as  angular  distance 
between  the  center  of  the  HMD  FOV  and  target.  Also,  target 
identification  information  can  be  included  in  the  TLL 
mechanization.  The  basic  TLL  indicates  the  combined 
azimuth  and  elevation  to  a  POI,  but  it  does  not  show  the 
angular  distance  to  the  same  point.  Angular  distance  is 
indicated  in  the  HUD  via  a  digital  readout  attached  to  the 
locator  line. 


The  HUD  TLL  is  mechanized  to  indicate  the  location  of 
the  POI  relative  to  the  nose  of  the  aircraft.  The  HMD  resident 
TLL  is  mechanized  to  indicate  the  location  of  the  POI  relative 
to  the  nose  on  the  pilot’s  face.  This  “look-to”  TLL  orientation 
has  to  indicate  angular  distance  changes  relative  to  head 
movement  rates  of  change  compared  to  the  much  slower  rates 
produced  either  by  target  or  ownship  maneuvering.  The  look- 
to  line  frame  of  reference  causes  it  to  be  affected  by  any 
individual  or  combination  of  ownship,  target,  and/or  head 
movement.  Given  that  the  head  can  move  up  to  800  degrees 
per  second,  a  digital  depiction  of  angular  distance  is  not 
appropriate.  Figure  1  is  an  example  of  how  the  digital 
mechanization  would  appear  if  the  convention  were  applied  to 
a  HMD.  An  analog  display  of  angular  distance  is  better  suited 
for  the  look-to  oriented  TLL  because  it  indicates  rate  and  trend 
information,  but  the  accuracy  of  the  digital  information  is 
absent.  Digital-like  accuracy  and  analog  trend  features  are 
both  desirable  because  the  task  dependent  use  of  the 
symbology  can  support  both  ballistic  head  movement  and 
continuous  tracking  behavior.  Figure  2  shows  a  TLL  with  only 
analog  angular  distance  information.  In  this  example,  the 
arrow  grows  or  shrinks  as  a  function  of  angular  distance.  The 
longer  the  line,  the  farther  the  POI  is  from  the  center  of  the 
display  FOV.  Alternatively,  Figure  3  shows  a  reflected  cue 
TLL  designed  to  support  instantaneous  interpretation  as  well 
as  continuous  tracking  behavior.  Additional  information  can 
be  added  to  this  design  by  shape  coding  the  cue  to  indicate 
target  identification  or  some  other  meaning. 

The  “reflected”  cue  slides  along  the  line  relative  to 
changes  in  angular  distance.  In  this  case,  the  reflected  cue 
moves  toward  the  edge  of  the  display  FOV  as  the  HMD  FOV 
and  POI  converge.  This  mechanization  ensures  that  the 
observer’s  attention  is  drawn  toward  the  area  where  the  POI 
will  appear  instead  of  toward  the  center  of  the  display  FOV. 
The  reflected  cue  also  acts  as  a  instantaneous  indicator  of 
angular  distance  by  its  relationship  to  a  constant  radius  locator 
line.  The  length  of  the  drawn  line  represents  the  angular 
distance  beyond  the  display  drawing  surface.  The  location  of 
the  cue  on  line  shows  the  relationship  of  the  POI  to  the  center 
of  the  display  FOV  and  the  area  beyond  the  display.  For 
instance,  with  omni-directional  tracking  capability,  the  POI 
can  be  located  up  to  1 80  degrees  from  the  center  of  the  display 
FOV.  The  locator  line  is  used  to  represent  this  180  degrees 
minus  its  own  subtended  angle  (the  display  drawing  surface  is 
equal  to  the  radius  of  the  display  FOV).  In  a  case  where  the 
locator  line  subtends  10  degrees  (20  degree  FOV),  the  line 
represents  the  170  degrees  beyond  the  display.  The  location  of 
the  reflected  cue  at  the  halfway  point  on  the  drawn  line 
represents  a  instantaneous  angle  of  85  degrees  from  the  center 
of  the  display  FOV  (170/2  =  85).  Empirical  studies  have 
shown  this  mechanization  to  support  improved  target  search 
and  tracking  performance  (Geiselman  and  Tsou,  1996;  Craig, 
Marshall,  and  Jordan,  1997). 

The  following  HMD  resident  locator  line  design 
principles  have  been  derived  from  previous  research  and 
operational  experience: 

1)  There  should  be  only  one  TLL  presented  at  any  one  time 
within  the  HMD  FOV. 
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Figure  1 .  TLL  with  digital  angular  distance  indication. 
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Figure  2.  TLL  with  analog  angular  distance  indication. 


Figure  3.  TLL  with  reflected  cue  angular  distance  indication. 

2)  The  mechanization  of  the  TLL  should  be  look-to  oriented. 

3)  The  TLL  should  be  anchored  to  the  center  of  the  HMD 
FOV.  This  is  intended  to  give  the  observer  a  consistent 
ability  to  locate  the  line  symbology  among  display  and 
background  clutter. 

4)  The  target  locator  line  should  maintain  a  constant  radius 
within  the  HMD  FOV.  This  will  support  both  the  anchor 
principle  as  well  as  indicate  the  edge  of  the  display  FOV. 

5)  Symbol  compression  should  be  used  to  indicate  the 
relative  angular  distance  between  the  HMD  FOV  and  the 
target  location. 

6)  Indications  of  closure  should  move  toward  the  edge  of  the 
FOV  (reflected  cue). 


HMD  OWNSHIP  STATUS  DISPLAY  DESIGN 

Because  HMD-equipped  pilots  tend  to  look  farther  off 
axis  for  longer  periods  of  time  versus  HUD-only  pilots,  it 
follows  that  the  HMD  should  contain  the  information  to  which 
the  pilots  would  otherwise  attend  when  they  look  forward  into 
the  cockpit.  Ownship  status  is  an  example  of  this  type  of 
information.  For  our  purposes,  ownship  status  information  is 
intended  to  keep  the  pilot  informed  of  the  primary  flight 
parameters  while  performing  targeting  tasks.  It  is  further 
intended  to  keep  the  pilot  from  becoming  spatially  disoriented 
and  is  not  intended  to  be  used  to  recover  from  incidences  of 
spatial  disorientation.  Included  as  information  is  airspeed, 
altitude,  heading,  and  attitude.  This  information  is  most  useful 
during  low  light  and  degraded  visual  conditions.  The 
challenge  to  the  designer  is  to  develop  symbology  that 
supports  the  information  objectives  without  adversely 
affecting  the  primary  use  of  the  HMD.  This  means  the 
symbology  has  to  be  highly  usable  for  the  associated  cost  and 
clutter.  It  is  not  acceptable  that  HMD  ownship  information 
cause  spatial  disorientation  or  any  other  significant  detraction 
from  the  primary  objective  of  the  information  source. 

The  following  are  general  ownship  status  symbology 
design  principles  which  have  been  derived  from  empirical 
research  and  flight  test  feedback.  Included  are  two  figures 
intended  to  represent  symbol  set  designs  both  in  (Figure  4) 
and  not  in  (Figure  5)  compliance  with  the  following 
principles. 

1)  Off-boresight  ownship  status  information  should  be 
included  in  the  HMD  FOV  any  time  visual  conditions  are 
less  than  day  visual  meteorological  conditions. 

2)  The  information  should  include  ownship  airspeed,  altitude, 
heading,  attitude,  and  possibly  acceleration  and  vertical 
velocity. 

3)  The  purpose  of  off-axis  ownship  information  is  to  keep  the 
pilot  aware  of  state  changes  in  support  of  the  primary 
HMD  tactical  functions.  The  display  is  intended  to  keep 
the  pilot  from  entering  an  unusual  attitude  vs.  allowing  the 
pilot  to  recover  from  an  unusual  attitude  while  performing 
off-boresight  viewing. 

4)  To  reduce  clutter  within  the  small  HMD  FOV,  it  is 
acceptable  to  use  digital  information.  This  supports  the 
reduction  of  attentional  capture. 

5)  Digital  information  should  be  used  sparingly  and  in  such  a 
way  that  its  spatial  location  helps  convey  meaning  or 
identification.  For  instance,  airspeed  should  be  located  left 
of  altitude  and  heading  should  be  displayed  between 
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airspeed  and  altitude.  Figure  4  shows  digital  information 
used  to  form  the  ownship  reference  symbol  for  the  attitude 
symbology.  The  basic  “T”  primary  flight  information 
format  convention  is  maintained. 

6)  Ownship  information  should  be  kept  in  close  proximity  to 
other  ownship  information.  The  figures  demonstrate  the 
effect  of  non-distributed  information  (Fig.  4)  vs. 
distributed  information  (Fig.  5).  This  is  done  to  reduce 
clutter  and  promote  space-based  attention.  In  Figure  5, 
clutter  is  spread  across  the  display  surface  and  a  wide  scan 
pattern  is  required  to  sample  the  required  information. 

7)  To  reduce  occlusion  of  POI  in  the  outside  world,  ownship 
type  information  should  be  located  in  the  bottom  portion  of 
the  HMD  FOV  for  the  air-to-air  application  and  in  the  top 
portion  of  the  FOV  for  air-to-ground  applications. 

8)  Attitude  information  should  support  maneuvering 
throughout  the  aircraft  performance  envelope  for  each  axis. 
The  attitude  display  or  ownship  information  set  should  not 
be  limited  relative  to  the  aircraft  capability,  and  roll,  pitch, 
and  yaw  movements  should  be  indicated.  The  attitude 
reference  in  Figure  4  represents  a  total  180  degrees  of 
climb/dive  (C/D)  angle  compared  to  the  total  of  5  degrees 
C/D  angle  represented  in  Figure  5. 

9)  To  reduce  clutter  and  maximize  usability,  the  attitude 
reference  should  be  compressed  (a  ratio  of  visual  angle 
subtended  by  the  symbology  to  the  angle  represented  by 
the  symbology).  This  allows  aircraft  maneuvering  to  be 
displayed  via  a  global  display.  High  compression  ratios 
promote  enhanced  interpretation  by  slowing  the  apparent 
motion  of  dynamic  effects,  but  they  reduce  precision.  The 
attitude  reference  in  Figure  4  represents  a  total  180  degrees 
of  C/D  angle  compared  to  the  total  of  5  degrees  C/D  angle 
represented  in  Figure  5. 
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Figure  4.  Non  distributed  ownship  status  symbology. 
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Figure  5.  Distributed  ownship  status  symbology. 

10)  The  HMD  attitude  reference  should  be  flight-path  based. 
This  mechanization  gives  the  most  meaningful  account  of 
the  aircraft  energy  state  and  instantaneous  changes  in 
altitude. 

11)  Attitude  information  should  be  forward  referenced.  This  is 
done  to  reduce  the  potential  of  disorientation  caused  by 
coupling  head  and  aircraft  movement  with  attitude  display 
changes.  Forward  referenced  displays  are  also  easier  to 
“look  around”  versus  information  that  is  superimposed 
over  the  outside  world.  Additionally,  forward  referenced 
displays  are  not  affected  by  tracker  system  lags  and  delays. 

12)  The  observer  perspective  interpretation  should  be  inside- 
out. 

EVALUATION  METHODOLOGY 

The  following  are  recommended  HMD  symbology 
evaluation  methodology  guidelines  intended  to  be  used  for 
symbology  design  principle  validation  purposes.  Similar  to  the 
symbology  development,  held  constant  across  the 
methodology  design  is  the  belief  that  the  primary  purpose  of 
the  HMD  is  target  cueing.  Therefore,  representative  evaluation 
tasks  are  those  which  include  off-axis  target  searching, 
designating,  and  tracking.  Evaluation  methodologies  should  be 
designed  to  be  flexible  so  future  candidate  symbologies  and 
other  interface  technologies,  such  as  multi-sensory  displays, 
can  be  reliably  compared  to  previously  collected  data.  A 
second  major  objective  is  to  develop  a  methodology  that  is 
both  empirically  and  operationally  valid.  It  should  be 
experimentally  controlled  but  recognized  by  subject  matter 
experts  as  operationally  relevant.  The  methodology  should 
include  the  following  features  (see  Geiselman  et  al,  1998): 
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1)  A  multi-phased  trial  approach  should  be  used  to  help 
ensure  trial  continuity.  Each  trial  should  be  formed  of 
separate  phases  which  are  treated  and  analyzed  as 
separate  tasks. 

2)  A  dual  task  paradigm  should  be  employed  with  off-axis 
targeting  (search,  location,  designation,  and  tracking) 
primary  tasks. 

3)  The  secondary  tasks  include  flight  tasks  such  as  attitude 
maintenance,  maneuvering,  and  extreme  attitude 
maneuvering. 

4)  Because  of  the  operational  nature  of  the  tasks,  at  least  the 
initial  evaluations  should  use  subject  matter  experts  as 
experimental  subjects. 

5)  Independent  variable  manipulations  should  include 
symbology  format  type,  a  no  HMD  symbology  baseline 
condition,  and  natural  horizon  presence  (on  or  off  to 
simulate  non-degraded  and  degraded  visual  conditions). 

6)  Measurement  metrics  should  include  task  perfonnance, 
subject  behavior  (head  movement),  and  subjective 
feedback.  Subjective  feedback  should  include  preference 
questionnaires,  workload  estimates,  and  situation 
awareness  ratings. 


CONCLUSION 

The  near  term  (within  five  years)  AFRL  goal  is  to  develop 
a  performance-optimized  HMD  symbology  set  intended  for 
present  technology.  Toward  this  end,  the  approach  includes 
the  identification  of  empirically  derived  design  principles 
which  can  be  applied  across  diverse  HMD  applications.  The 
resulting  symbology  will  form  the  baseline  design  for  the  first 
operationally  fielded  HMDs.  Likewise,  the  symbology  formats 
will  be  included  in  design  guidance  documentation  for  related 
applications.  After  the  baseline  symbology  set  is  established, 
new  designs  will  be  added  or  will  replace  baseline  designs 
whenever  operational  need,  technological  innovation,  or 
demonstrated  performance  enhancement  warrants  the  change. 
Regardless  of  the  eventual  HMD  symbology  format,  the 
derived  design  principles  should  help  ensure  that  the 
symbology  functionality  is  consistent  across  applications.  A 
somewhat  standardized  evaluation  methodology  will  help 
ensure  that  reliable  comparisons  of  various  symbol  set 
candidates  are  made. 
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